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follow-ups to a significant omnibus F test. This study compared observed Type 
I errors with nominal alphas of 0.01, 0.05, and 0.10 compared for various 

sample sizes and numbers of groups. Monte Carlo methods were used to generate 
replications expected to provide 0.95 confidence intervals of + /- 0.001 
around the nominal alphas of 0.10, 0.05, and 0.01 for 42 combinations of n 
(5, 10, 15, 20, 30, 60, and 100) and numbers of groups (3, 4, 5, 6, 8, and 
10) . Means and standard deviations of observed Type I error rates and 
percentages of observed Type I errors falling below, within, and above the 
0.95 confidence intervals were determined for total number of Type I errors. 
The results indicate that HSD is conservative relative to experimentwise Type 
I error control across all alpha levels, sample sizes, and number of groups. 
However, when per-experiment (total Type I errors) is of interest, HSD was 
liberal at alpha of 0.10 and 0.05, but was very conservative when alpha was 
0.01. Results also point out the differences inherent in selection of a Type 
I error mode of control. Differences between per-experiment and 
experimentwise Type I error control was mostly a function of the number of 
groups being compared. As the number of groups increased, the difference 
between per-experiment and experimentwise error proportions increased. 
However, sample size was also a significant predictor; as sample size 
increased, the difference decreased. (Contains 9 tables and 14 references.) 
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Abstract 



Tukey’s HSD is probably the most recommended and used procedure for 
controlling Type I error-rate when making multiple pairwise comparisons as follow-ups 
to a significant omnibus F test. The purpose of this study was to compare observed Type 
I errors with nominal alphas of .01, .05, and . 10 for various sample sizes and numbers of 
groups. Monte Carlo methods were used to generate replications expected to provide .95 
confidence intervals of +/- .001 around the nominal alphas of .10, .05, and .01 for 42 
combinations of n (5, 10, 15, 20, 30, 60, and 100) and numbers of groups (3, 4, 5, 6, 8, 
and 10). Means and standard deviations of observed Type I error-rates and percentages 
of observed Type I errors falling below, within, and above the .95 Cl’s were determined 
for total number of Type I errors. The results indicate HSD is conservative relative to 
experimentwise Type I error control across all alpha levels, sample sizes, and number of 
groups. However when per-experiment (total Type I errors) is of interest, HSD was 
liberal at alpha of . 10 and .05, but was very conservative when alpha was .0 1 . Results 
also point out the differences inherent in selection of a Type I error mode of control. 
Differences between per experiment and experimentwise Type I error control was mostly 
a function of the number of groups being compared. As the number of groups increased, 
the difference between per-experiment and experimentwise error proportions increased. 
However, sample size was also a significant predictor; as sample size increased the 
difference decreased. 
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THE TUKEY HONESTLY SIGNIFICANT DIFFERENCE PROCEDURE 
AND ITS CONTROL OF THE TYPE I ERROR-RATE 




Whenever a researcher has more than two comparisons to test, control of the Type 
I error-rate becomes a concern. Soon after Fisher developed the process of analysis of 
variance (ANOVA), he recognized the potential problem of the error-rate becoming 
inflated when multiple t-tests were performed on three or more groups. He discusses this 
problem in the 1935 edition of his famous book. The Design of Experiments. His 
recommendation of using a more stringent alpha when performing his Least Significant 
Difference Procedure (LSD) is based on this concern. However, researchers still 
criticized the LSD as providing inadequate control of Type I error. This early recognition 
of the problem has resulted in hundreds of multiple comparison procedures being 
developed over the years. One of the more popular of these procedures (Toothaker, 

1993) has been the Honestly Significance Difference Procedure (HSD). 

Most studies of Type I error rates for follow-up of pairwise mean differences have 
been based on what is referred to as experimentwise or familywise error control 
philosophies. These terms are credited to Miller (1966). Experimentwise (EW) Type I 
error relates to finding at least one significant difference by chance for the specified alpha 
level. In these cases, the only difference of concern is the largest mean difference. 
Experimentwise Type I error control ignores the possibility of multiple Type I errors in 
the same experiment. The pairwise mean differences for those other than the largest 
mean difference are not considered. Type I error control is such that not all possible 
Type I errors are evaluated. In these cases, many procedures such as Tukey’s HSD are 
considered to have conservative Type I error control since the actual probabilities of 
finding at least one Type I error are lower than the nominal alpha level. 

Per-experiment (PE) Type I error control considers all the possible Type I errors 
that can occur in a given experiment. Thus, more than one Type I error per experiment is 
possible and reasonably likely to occur if there is an experimentwise Type I error on the 
highest mean difference. Klockars and Hancock (1994) pointed out the importance and 
risks associated with this distinction. They found, using a Monte Carlo simulation, that 
there was a difference of .0132 in the per-experiment and experimentwise Type I error 
rates for Tukey’s HSD when alpha was set at .05. This discussion was expanded in their 
1996 review titled “The Quest for a” (Hancock & Klockars). Thus, when one has exact 
control of Type I error in the experimentwise situation, the per-experiment Type I error 
probability is higher. One of the purposes of this research was to examine how much of a 
difference there may be between experimentwise and per-experiment Type I error rates 
for Tukey’s HSD when used with alpha levels of .10, .05, and .01 and to determine the 
relative influence on this difference of number of groups and number of subjects per 
group. It is predicted that as the number of groups increases, the difference between 
experimentwise and per-experiment error rates will increase since there are more 
pairwise differences in the set. While most Type I error research is based on an 
experimentwise mode, the per-experiment Type I error is more consistent with the reality 
of pairwise hypothesis testing. It is not only the largest mean difference subjected to 
error control, but all the pairwise differences. Thus, we favor a per-experiment mode of 
Type I error control. 
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Background 



The earliest example of what we now know as a multiple comparison procedure 
could be found in 1929, when Working and Hotelling applied simultaneous confidence 
intervals to regression lines. The Fisher (1935) reference cited earlier was the first 
application to the process of ANOVA. The Type I error-rate control problem was also 
referred to by Pearson and Sekar in 1936 and Newman in 1939. Newman described a 
multiple comparison test that used the “Studentized Range Statistic.” It is said that his 
work was prompted by a discussion he had with Student. Years later, Keuls published an 
updated version of the procedure (1952) using the Studentized range. We now know that 
multiple comparison procedure as the Student-Newman-Kuels Procedure. 

Probably the most-used multiple comparison procedure is the Honestly 
Significance Difference (HSD) (Toothacker, 1993). Tukey’s Honestly Significant 
Difference Procedure (HSD) was presented originally in an non-published paper by 
Tukey in 1953. Based on the Studentized range statistic originally derived by Gosset 
(a.k.a., Student) (1907-1938), the original derivation assumed normality, homogeneity of 
variance, and an equal number of observations per group. The Studentized range 
statistic, unlike the t-statistic, takes into account the number of means being compared, 
adjusting for the total number of tests to make all pairwise comparisons (Kennedy & 
Bush, 1985). Purportedly, the HSD gives a per-experiment error rate. Many common 
statistical texts recommend it be used as a protected test for all post-hoc tests (e g., 

Hayes, 1988; Kennedy & Bush, 1985; Kirk, 1982; Maxwell & Delaney, 1990). Doing 
so, it is supposed to control the experimentwise error at a preselected value such as .05 or 
.01. 



Later refinements of the HSD demonstrated its robustness to violations of the 
normality and homogeneity assumptions. Replacing the sample size of the groups in the 
formula with the harmonic mean of the sample sizes has been described (Kirk, 1982) as 
an alternative when the sample sizes are not equal. In any case, this study uses only cases 
where the sample sizes are equal among groups and the other assumptions are achieved. 



Methodology 

Monte Carlo methods were used to generate the data for this research. All data 
comprising the groups whose means were compared were generated from a random 
normal deviate routine, which was incorporated into a larger compiled QB ASIC program 
that conducted all needed computations. The program was written by the senior author. 
All sampling and computation, conducted with double-precision, routines were verified 
using S AS® programs. The program was run on a Dell Pentium II, 266 MHZ personal 
computer. 

Several sample size and number of groups arrangements were selected to give a 
range of low, moderate, and large case situations. The number of groups was: 3, 4, 5, 6, 
8, and 10 and the sample sizes for each group were: 5, 10, 15, 20, 30, 60, and 100, which 
when crossed gave 42 experimental conditions. This was replicated for three nominal 
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alphas of .10, .05, and .01. The approach used was to determine what number of 
replications would be needed to provide an expected .95 confidence interval of +/- .001 
around the nominal alpha. This is an approach to examination of how well observed 
Type I error proportions are reasonable estimates of a standard nominal alpha. In other 
words, if alpha is the standard, what proportion of the estimates of actual Type I error 
proportions can be considered accurate, as evidenced by them being within the expected 
.95 confidence interval around nominal alpha? 

This was based on the assumption that errors would be normally distributed 
around the binomial proportion represented by nominal alpha. Thus, when alpha was .10, 
345742 replications were needed to have a .95 confidence interval of +/- .001 or between 
.099 and .101. When alpha was .05, 182475 replications were needed to have a .95 
confidence interval of +/- .001 or between .049 and .051 and when alpha was .01, 38032 
replications were needed to have a .95 confidence interval of +/- .001 or between .009 
and .011. Observed Type I error proportions falling into the respective .95 confidence 
intervals are considered to be reliable estimates of the expected Type I error rate. 
Observed Type I error proportions falling below the .95 Cl are considered to be 
conservative, and observed Type I error proportions falling above the .95 Cl are 
considered to be liberal. 

Within each nominal alpha/sample size/number of groups configuration, the 
number of ANOVA replications were generated. Each replication involved drawing of 
elements of the sample from a distribution of normal deviates, computation of sample 
means, and the omnibus F test. If the omnibus F test was significant, Tukey’s HSD was 
computed and all pairwise mean differences were compared with the HSD critical value. 
While HSD uses only one critical value for all differences, the pairwise differences were 
recorded in a hierarchical fashion to determine pairwise differences significant at each of 
the numbers of steps between means from K down to 2. This approach permitted 
determination of experimentwise Type I error (at least one Type I error per experiment) 
or a Type I error for the largest mean difference, and per-experiment Type I errors or the 
total number of Type I errors observed regardless of where they are in the stepwise 
structure. 

Summary statistics were computed for each alpha level for experimentwise and 
per-experiment conditions including: the mean proportion of Type I errors, standard 
deviation of the proportion of Type I errors, minimum proportion, maximum proportion, 
and the percentage of proportions falling in the three regions associated with the .95 
confidence interval. Additional analysis included computation of differences between 
per-experiment proportions and experimentwise proportions (PE-EW) and 
correlation/regression analysis to determine relative influences of number of groups and 
sample sizes on this difference. 



Results 

The results were presented separately for alphas of .10, .05, and .01. In each case, 
the trends will be discussed and a conclusion stated. 
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Type I errors when alpha= .10 

Tables 1 and 2 present the proportions and summary statistics of Type I errors 
when experimentwise and pre-experiment modes are used for the varying number of 
groups and sample sizes. The mean proportion of Type I errors for the experimentwise 
mode was .081348 and 100% of the 42 mean proportions were below the expected .95 
confidence interval. Thus, in this case HSD was conservative to making Type I errors. 
HSD became more conservative relative to experimentwise Type I errors as the number 
of groups increased. 

However, when per-experiment rate is considered, the mean proportion of Type I 
errors was .127948 and 100% of the observed proportions were above the expected .95 
confidence interval, indicating a liberal condition. As the number of groups increased, 
HSD became more liberal. The difference between the per-experiment and 
experimentwise proportions, as indicated in Table 3, was .04660. The correlation of 
number of groups with difference was .8899 (g < .0001) and the correlation of sample 
size and difference was -.2213 (g = .1590). These were entered into a regression equation 
and the first variable entered (number of groups) accounted for 79.2% of the variance; the 
addition of the sample size variable increased the variance-accounted-for percentage to 
84. 1. Clearly, in this case, the number of groups was the major factor associated with this 
difference. 

Type I errors when alpha= .05 

Tables 4 and 5 present the proportions and summary statistics of Type I errors 
when experimentwise and pre-experiment modes are used for the varying number of 
groups and sample sizes. The mean proportion of Type I errors for the experimentwise 
mode was .038655 and 100% of the 42 mean proportions were below the expected .95 
confidence interval. Thus, in this case HSD was conservative to making Type I errors 
and tended to become more conservative as the number of groups increased. 

However, when per-experiment rate is considered, the mean proportion of Type I 
errors was .055310 and 97.6% of the observed proportions were above the expected .95 
confidence interval, indicating a liberal condition. The tendency to become more liberal 
as the number of groups increased was not as apparent as it had been in the alpha= .10 
condition. The difference between the per-experiment and experimentwise proportions, 
as indicated in Table 6, was .01665. The correlation of number of groups with difference 
was .8266 (g < .0001) and the correlation of sample size and difference was -.2833 (g = 
.0690). These were entered into a regression equation and the first variable entered 
(number of groups) accounted for 68.3% of the variance; the addition of the sample size 
variable increased the variance-accounted-for percentage to 76.4. Clearly, in this case, 
the number of groups was the major factor associated with this difference. However, the 
relative influence decreased for number of groups and increased for sample size when 
compared with the alpha equal to .10 condition. 
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Type I errors when alpha= .01 

Tables 7 and 8 present the proportions and summary statistics of Type I errors 
when experimentwise and pre-experiment modes are used for the varying number of 
groups and sample sizes. The mean proportion of Type I errors for the experimentwise 
mode was .007019 and 97.6% of the 42 mean proportions were below the expected .95 
confidence interval. Thus, in this case HSD was conservative to making Type I errors 
and tended to become more conservative as the number of groups increased. 

However, when per-experiment rate is considered, the mean proportion of Type I 
errors was .008772 and 57. 1% of the observed proportions were below the expected .95 
confidence interval while 42.9% were within the .95 confidence interval. Conditions of 
small numbers of groups and/or small sample sizes were more likely to have proportions 
within the .95 confidence interval while conditions with higher numbers of groups and 
larger sample sizes tended to have proportions falling below the .95 confidence internal 
or being conservative. 

The difference between the per-experiment and experimentwise proportions, as 
indicated in Table 9, was .001753. The correlation of number of groups with difference 
was .6588 (p < .0001) and the correlation of sample size and difference was -.4091 (p = 
.0071). These were entered into a regression equation and the first variable entered 
(number of groups) accounted for 43.4% of the variance; the addition of the sample size 
variable increased the variance-accounted-for percentage to 60. 1 . Clearly, in this case, 
the number of groups was the major factor associated with this difference. However, the 
relative influence decreased for number of groups and increased for sample size when 
compared with the alpha equal to .10 and .05 conditions. 



Conclusions 

In general, Tukey’s HSD does not provide for accurate Type I error control either 
when experimentwise or per-experiment is the control philosophy. Its accuracy is a 
function of alpha, number of groups, sample sizes, and control philosophy. As it has 
been used in most applications, it provides for conservative experimentwise control and 
liberal per-experiment control. Tukey’s HSD is conservative if experimentwise Type I 
error is the philosophy regardless of alpha, number of groups, or sample sizes when 
conducted as a protected test. 

If per-experiment is the Type I error philosophy, HSD is liberal if alpha is . 10 or 
.05; but when alpha is .01, it is accurate when few groups or small samples are used and 
tends to become conservative when more and larger samples are used. The difference 
between experimentwise and per-experiment proportions of Type I errors is large enough 
to be a concern in having confidence that differences are not likely to be a function of a 
pre-specified level of chance, particularly when alpha is higher. 

The most important determinate of the difference between experimentwise and 
pre-experiment Type I error proportions is the number of groups. As the number of 
groups increases, the difference increases. The influence of sample size is inverse: as 
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sample size increases, the difference between experimentwise and pre-experiment Type I 
error proportion decreases. As alpha decreases, the influence of number of groups 
decreases while the influence of sample size increases. 

Additional study of the discrepancy between experimentwise and per-experiment 
Type I errors is needed. We need to determine just how important this discrepancy is. 

The current study did not consider the case of unequal sample sizes or heterogenous 
variances. Is it the same under conditions of unequal sample sizes and/or variances? 

The determination of factors relating to the discrepancy remains to be done. The current 
study only included the HSD procedure conducted as a protected test. Does this 
phenomenon occur with other MCPs? The determination of why these patterns change as 
alpha changes also remains to be done. 

Ultimately, there remains a need to identify or develop a multiple comparison 
procedure that will provide true per-experiment Type I error control for examining 
pairwise differences under a number of conditions. This study has demonstrated that the 
Monte Carlo method might be an effective means of doing so. 
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Table 1 . Observed Experimentwise Type I Errors for HSD, Alpha= . 10, CI. 95 - +/- .001 



K 


n= 5 


n= 10 


n= 15 


n= 20 


O 

m 

11 

c 


n= 60 


n =100 


3 


.0932 i 


.0924 i 


.0925 i 


.0923 i 


.0916 -l 


.0918 i 


.0917 -l 


4 


.0882 i 


.0879 i 


.0877 i 


.0874 i 


.0870 i 


.0866 i 


.0873 i 


5 


.0849 i 


.0840 1 


.0835 i 


.0839 i 


.0829 i 


.0829 i 


.0827 i 


6 


.0814^ 


.0801 i 


.0802 i 


.0797 i 


.0798 i 


.0794 i 


.0797 i 


8 


.0770 i 


.0753 i 


.0742 i 


.0740 i 


.0733 i 


.0734 i 


.0743 i 


10 


.0727 i 


.0712 


.0703 1 


.0702 1 


.0689 i 


.0689 i 


.0702 i 


i indicates p below Cl 95 


BOLD indicates p within CI. 9 J 


t indicates p above Cl 95 


Mean 


SD 


Min. 


Max. 


%i 


% within 


%t 


.081348 .007553 


.0689 


.0932 


100 


0 


0 


Table 2. Observed Per-Experiment (Total) Type I Errors for HSD, Alpha= 
CI. 95 = +/- 001 


. 10 , 


K 


n= 5 


n= 10 


n= 15 


n= 20 


O 

cn 

It 

C 


n= 60 


% 

O 

O 


3 


.1217 t 


.1 160 t 


.1151 t 


.1143 t 


.1132 t 


.1132 T 


.1125 t 


4 


.1335 t 


.1252 t 


.1236 t 


.1221 t 


.1208 t 


.1198 t 


.1200 t 


5 


.1411 t 


.1307 t 


.1279 t 


.1268 t 


.1246 t 


.1238 t 


.1227 t 


6 


.1455 t 


.1330 t 


.1305 t 


.1283 t 


.1266 t 


.1263 t 


.1256 T 


8 


.1510 t 


.1374 t 


.1317 t 


.1308 t 


.1274 t 


.1267 t 


.1283 T 


10 


.1516 t 


.1369 t 


.1317 T 


.1516 t 


.1297 t 


.1259 t 


.1287 t 


i indicates p below Cl 95 


BOLD indicates p within CI. 9 J 


T indicates p above Cl 95 


Mean 


SD 


Min. 


Max. 


%i 


% within 


%T 


.127948 .009794 


.1125 


.1516 


0 


0 


100.0 
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Table 3. Differences in Per-Experiment and Experimentwise Type I Errors for HSD, 
Alpha= .10 



K 


n= 5 


n= 10 


n= 15 


n= 20 n= 30 n= 60 


n =100 


Mean 


3 


.0285 


.0236 


.0226 


.0220 


0216 .0214 


.0208 


.0229 


4 


.0453 


.0373 


.0359 


.0347 


0338 .0332 


.0327 


.0361 


5 


.0562 


.0467 


.0444 


.0429 


0417 .0409 


.0400 


.0447 


6 


.0641 


.0529 


.0503 


.0486 


0468 .0469 


.0459 


.0508 


8 


.0740 


.0621 


.0575 


.0568 


0541 .0533 


.0540 


.0588 


10 


.0789 


.0657 


.0614 


.0814 


0608 .0570 


.0585 


.0662 


Mean 


.0578 


.0481 


.0454 


.0477 


0431 .0421 


.0420 


.0466 


Table 4 


. Observed 


Experimentwise Type I Errors for HSD, Alpha= 


: .05, Cl 95 - 


.001 


K 


n= 5 


n= 10 


n= 15 


n= 20 


O 

II 

c 


n= 60 


n =100 


3 


.0461 i 


.0455 i 


.0455 i 


.0452 i 


.0453 i 


.0446 i 


.0451 i 


4 


.0434 i 


.0422 i 


.0419 -i 


.0421 i 


.0427 i 


.0418 


.0422 i 


5 


.0404 i 


.0401 i 


.0398 i 


.0399 i 


.0389 i 


.0397 i 


.0401 i 


6 


.0394 i 


.0379 i 


.0380 -l 


.0371 i 


.0368 i 


.0378 i 


.0380 i 


8 


.0364 i 


.0345 i 


.0339 i 


.0338 i 


.0339 i 


.0336 i 


.0347 i 


10 


.0337 i 


.0324 i 


.0321 i 


.0317 -l 


.0319 -l 


.0313 i 


.0321 i 



i indicates p below Cl 95 BOLD indicates p within Cl 95 T indicates p above Cl 95 



Mean 


SD 


Min. 


Max. 


%i 


% within 


% t 


.038655 


.004578 


.0313 


.0461 


100 


0 


0 
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Table 5. Observed Per-Experiment (Total) Type I Errors for HSD, Alpha- .05, 
CI.95=+/- 001 



K 


n= 5 


n= 10 


n= 15 


n= 20 


n= 30 


n= 60 


n=100 


3 


.0573 t 


.0542 T 


.0541 t 


.0532 T 


.0532 T 


.0524 T 


.0523 T 


4 


.0612 t 


.0559 T 


.0548 t 


.0547 T 


.0548 T 


.0536 T 


.0538 T 


5 


.0621 t 


.0573 t 


.0565 T 


.0554 t 


.0537 T 


.0542 T 


.0544 T 


6 


.0640 t 


.0574 t 


.0562 T 


.0543 T 


.0532 T 


.0546 T 


.0544 t 


S 


.0638 t 


.0573 t 


.0551 T 


.0539 t 


.0530 T 


.0522 T 


.0533 t 


10 


.0635 t 


.0557 t 


.0533 T 


.0533 t 


.0528 T 


.0504 


.0522 t 



i indicates p below Cl 95 BOLD indicates p within Cl 95 T indicates p above Cl 95 



Mean 


SD 


Min. 


Max. 


%i 


% within 


%T 


.055310 


.003240 


.0504 


.0640 


0 


2.4 


97.6 



Table 6. Differences in Per-Experiment and Experimentwise Type I Errors for HSD, 
Alpha= .05 



K 


n= 5 


n= 10 


n= 15 


n= 20 


O 

II 

G 


n= 60 


S3 

O 

O 


Mean 


3 


.0112 


.0087 


.0086 


.0080 


.0079 


.0078 


.0072 


.0085 


4 


.0178 


.0137 


.0129 


.0126 


.0121 


.0118 


.0116 


.0132 


5 


.0217 


.0172 


.0167 


.0155 


.0148 


.0145 


.0143 


.0164 


6 


.0246 


.0195 


.0182 


.0172 


.0164 


.0168 


.0164 


.0184 


8 


.0274 


.0228 


.0212 


.0201 


.0191 


.0186 


.0186 


.0211 


10 


.0298 


.0233 


.0212 


.0216 


.0209 


.0191 


.0201 


.0223 


Mean 


.0221 


.0175 


.0165 


.0158 


.0152 


.0148 


.0147 


.0164 




12 



13 



Table 7. Observed Experimentwise Type I Errors for HSD, Alpha= .01, 0.95= +/- .001 



K 


n= 5 


n= 10 


n= 15 


n= 20 


O 

m 

II 

c: 


n= 60 


n=100 


3 


.0089 4 


.0085 4 


.0092 


.0080 4 


.0086 4 


.0086 4 


.0085 4 


4 


.0079 4 


.0083 4 


.0079 4 


.0085 4 


.0084 i 


.0076 4 


.0075 4 


5 


.0077 4 


.0069 4 


.0067 4 


.0073 4 


.0071 4 


.0073 4 


.0074 4 


6 


.0074 4 


.0070 4 


.0072 4 


.0064 4 


.0069 4 


.0062 4 


.0070 4 


8 


.0065 4 


.0059 4 


.0060 4 


.0060 4 


.0060 4 


.0057 4 


.0060 4 


10 


.0059 4 


.0053 4 


.0055 4 


.0055 4 


.0050 4 


.0052 4 


.0054 4 


4 indicates p below CI .95 


BOLD indicates p within CI .95 


T indicates p above CI .95 


Mean 


SD 


Min. 


Max. 


%4 


% within 


% t 


.007019 .001157 


.0050 


.0092 


97.6 


2.4 


0 



Table 8. Observed Per-Experiment (Total) Type I Errors for HSD, Alpha= .01, 
0.95= +/- .001 


K 


n= 5 


n= 10 


n= 15 


n= 20 


n= 30 


n= 60 


n=100 


3 


.0103 


.0095 


.0101 


.0090 


.0093 


.0094 


.0094 


4 


.0105 


.0096 


.0094 


.0103 


.0097 


.0088 4 


.0087 4 


5 


.0105 


.0083 4 


.0083 4 


.0091 


.0083 4 


.0087 4 


.0087 4 


6 


.0102 


.0092 


.0095 


.0075 4 


.0088 4 


.0077 4 


.0085 4 


8 


.0095 


.0082 4 


.0082 4 


.0078 4 


.0085 4 


.0073 4 


.0075 4 


10 


.0092 


.0078 4 


.0079 4 


.0080 4 


.0070 4 


.0069 4 


.0075 4 



4 indicates p below Cl 95 BOLD indicates p within CI .95 T indicates p above CI .95 



Mean 


SD 


Min. 


Max. 


%4 


% within 


% t 


.008772 


.000977 


.0069 


.0105 


57.1 


42.9 


0 
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Table 9. Differences in Per-Experiment and Experimentwise Type I Errors for HSD, 
Alpha= .01 



K 


II 

C 


o 

11 

c 


n= 15 


n= 20 


n= 30 


n= 60 


O 

o 

IT 

c 


Mean 


3 


.0014 


.0010 


.0009 


.0010 


.0007 


.0008 


.0009 


.0010 


4 


.0026 


.0013 


.0015 


.0018 


.0013 


.0012 


.0012 


.0016 


5 


.0028 


.0014 


.0016 


.0018 


.0012 


.0014 


.0013 


.0016 


6 


.0028 


.0022 


.0023 


.0011 


.0019 


.0015 


.0015 


.0019 


8 


.0030 


.0023 


.0022 


.0018 


.0025 


.0016 


.0015 


.0021 


10 


.0033 


.0025 


.0024 


.0025 


.0020 


.0017 


.0021 


.0024 


Mean 


.0027 


.0018 


.0018 


.0017 


.0016 


.0014 


.0014 


.0018 
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